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© Speech recognition apparatus which predicts word classes from context and words from word 
classes. 
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© A language generator for a speech recognition apparatus scores a word-series hypothesis by combining 
individual scores for each word in the hypothesis. The hypothesis score for a single word comprises a 
combination of the estimated conditional probability of occurrence of a first class of words comprising the word 
being scored, given the occurrence of a context comprising the words in the word-series hypothesis other than 
the word being scored, and the estimated conditional probability of occurrence of the word being scored given 
the occurrence of the first class of words, and given the occurrence of the context. An apparatus and method are 
provided for classifying multiple series of words for the purpose of obtaining useful hypothesis scores in the 
language generator and speech recognition apparatus. 
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The invention relates to speech recociii^" taT k " S,ia '" y " 15 m ' S ,nvention - 
which generate and score worries "^1?^? V* Chectaw ™* 0ther devi «* 

-C^^^ — — an d automatic 

Pr(W|Y) =- Pr < W ) p ; ( ^(^|W) £lj 

- one or more words, for example 
variable Y is an observed signal, and Pr(Y) the £22£ 0^,7"°° f WOrd ' Series "^thesis. The 
« the probability of occurrence of the I d series w ^ven °' ° CCUrrence of the ob served signal. Pr(W|Y) 
(Y|W) is the probability of occurrence of Z ob^'sZ^ Y n T" ° f ^ 0bSerVed Si9 " al Y " ^ 

For automatic speech recognition. Y i s^^n^^coustJc siqnal^See 6 ° Ccurr6nce of tne word-series W. 
Maximum Like.ihood Approach to Continuous SpJSrSSCL^Si 0 ; T' L " Bahl ' et " A 
and Machine Intelligence Vnh.m» pam,_ c No 2 M " h £ ^ Transactions on Pattern Analysis 

translation. Y is a sequence of words in another . a n. S?' P aaes 1^180.) For automatic language 
hypothesis. (See. fo? exampfe. Tf Bro^n e ^^825^ T th8 ' an9UaQe of the -rd-series 
Computational Linguistics . Vol. i« m» 2 June iflM M «J £ L PPr ° aCh T ° Machir > e Translation." 
sequence of characters produced by a pirtlj hSS ^2^1 aUt ° mati ? SPe " in9 C ° rreCtl '° n " Y is a 
Based Spelling Correction." h^m^^^^Z^ °' example ' E Ma ^ et al. "Context 

In all three appiications. ^irT a Z^n ^e sL^Z ZT -- °^ ^ Na 5 ' 1991 ' pa9es 517 - 52 2> 
gave rise to the signa. Y. In 'genera m^ ^Xen, wo S SET 9 * ^ ° f En9 " Sh W ° rdS ' W ' wh ' ch 
model minimizes the probably of choSn an Vr rZT Ser,e * can 9 ,ve nse to the same signal Y. The 
having the .argest conditional P^U^ £ S^^" 1 - * ** S °™ W 

the «aT oT^^ -f- series W given the observed sign, Y is 

the observed signal Y wi.l be produce when £e lortZLZ" >' T' tiplied by » the Probability that 
of observing the signal Y. word-ser.es W , s mtended. divided by (iii) the probability 

- «-c signa, Y given the 

language translation, the probability ofte sequeTce S^ST?- t ^ W ° rd SerieS W ' ,n automati ' 
English-translation word series W is es«maSd by 

automatic spelling correction, the probiO^a^^JL^TT ^ for the word seri «* W. In 
•mperfect typist given the hypothesized wo* series ^JLtf^ Y "l*"** by 3 p0Ssib,y 
word series W. es w ls est,nr >ated by using a mistyping model for the 

- - app.ca.on, the probaMty „ „. ^ ^ w ^ ^ ^ ^ ^ ^ 

Pr(w{).. Pr^) Pr(w 2 |w 1 ) ... <Pr <w k | w*" 1 ) , [2] 

so where wf represents a series of words w, , w 2 , ... k k 
In the conditional probability 
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is called the history or the predictor feature. Each word in the history is a predictor word. The term w k is 
5 called the predicted feature or the category feature. 

The mechanism for estimating the conditional probabilities in Equation 2 is called a language model. A 
language model estimates the conditional probabilities from limited training text. The larger the training text, 
and the larger the number of parameters in the language model, the more accurate and precise are the 
predictions from the language model. 
io In all three applications, the probability Pr(Y) of occurrence of the observed signal Y can either be 

modeled as a function of some parameters, or can be assumed to be independent of the word series W to 
be found. In the latter case, the term Pr(Y) is dropped out of Equation 1. 

It is an object of the invention to provide a language generating apparatus and method, such as for a 
speech recognition apparatus, having a highly accurate and precise language model with a relatively small 
75 number of parameters. 

It is another object of the invention to provide a language generating apparatus and method having 
highly accurate language model estimates of the conditional probabilities of predicted words from relatively 
small amounts of training text. 

It is a further object of the invention to provide an apparatus and method for classifying a set of 
20 observed events, such as a series of two or more words, for generating an accurate and precise language 
model. 

According to the invention, a language generator comprises means for generating two or more word- 
series hypothesis signals. Each word-series hypothesis signal represents a word-series hypothesis compris- 
ing a series of linguistic units. Each linguistic unit comprises one or more words selected from a language. 

25 The language generator further includes means for generating a single-unit hypothesis score signal for 
each linguistic unit in each word-series hypothesis. Each single-unit hypothesis score signal represents a 
single-unit hypothesis score comprising an estimate of the probability of occurrence of the individual 
linguistic unit in the word-series hypothesis. 

Means are provided for generating a word-series hypothesis score signal for each word-series 

30 hypothesis. Each word-series hypothesis score signal comprises a combination of the single-unit hypothesis 
scores for the linguistic units in the word-series hypothesis. 

The language generator further includes means for storing one or more candidate word-series 
hypothesis signals representing candidate word-series hypotheses having the best word-series hypothesis 
scores Output means are provided for outputting at least one word signal representing at least one word in 

35 at least one best candidate word-series hypothesis. 

In the language generator according to the present invention, the means for generating a single-unit 
hypothesis score signal for a first individual linguistic unit comprises means for estimating the conditional 
probability of occurrence of a first class of linguistic units comprising the first individual linguistic unit, given 
the occurrence of a context comprising the linguistic units in the word-series hypothesis other than the first 

40 individual linguistic unit. Means are also provided for estimating the conditional probability of occurrence of 
the first individual linguistic unit given the occurrence of the first class of linguistic units, and given the 
occurrence of the context. The single-unit hypothesis score signal for the first individual linguistic unit 
comprises a first component score comprising a combination of the estimated conditional probability of 
occurrence of the first class of linguistic units given the occurrence of the context, and the estimated 

45 conditional probability of occurrence of the first individual linguistic unit given the occurrence of the first 
class and given the occurrence of the context. 

In one aspect of the invention, the conditional probability of occurrence of the first individual linguistic 
unit given the occurrence of the first class of linguistic units, and given the occurrence of the context is 
estimated by the conditional probability of occurrence of the first individual linguistic unit given the 

so occurrence of the first class of linguistic units. 

The conditional probability of occurrence of the first class of linguistic units, given the occurrence of the 
context, may be estimated by the conditional probability of occurrence of the first class of linguistic units, 
given the occurrence of a first context set comprising one or more linguistic units in the word-series 
hypothesis other than the first individual linguistic unit. Alternatively, the conditional probability of occur- 

55 rence of the first class of linguistic units given the occurrence of the context may be estimated as a 
weighted combination of (a) the conditional probability of occurrence of the first class of linguistic units, 
given the occurrence of a first context set comprising one or more linguistic units in the word-series 
hypothesis other than the first individual linguistic unit, and (b) the conditional probability of occurrence of 
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linguistic unit. " " ac,,CB "/pomesis other than the first individual 

5 ^3;rr n rs% ?*- * «i in dividual linguistic unit may 

linguistic units comprising the ^^^X'^S' ° f "T^* ° f 3 SeCOnd c,ass <* 
linguistic units in the word-series hypoLte <i£ than thl r TTT ° f the C ° nteXt COm P risin 9 the 
provided for estimating the conditio^^ «* Means are'aiso 

the occurrence of the second class of linguistic unte anS oiven 1 « ,nd,V,dUa ' ' in9Ui5tiC Unit 9 iven 
>* component score is generated comprising a Cbinal'T 

occurrence of the second class of linguistic unit^ve^ « « t T** conditional Probability of 

conditional probability of occurrence J^J!^^^^^ ° f the COntext ' and th * ^timated 
class of linguistic units and given the JZ^^^J^Tl *~ ° f *• aaoond 

20 2£^2ss ir^-ssfLr of ,in9uistic »-* *« - ™ - 

units, given the occurrence of a 1st conterf set ° CCU,Tence of the «eoond Cass of linguistic 

hypothesis other than the first In^SX^^T^^ *» m6 unM » * the word ^ 
rence of the second class of linguistic uni s a ten T™^ conditional Probability of occur- 

comprisingoneormorelingu^ 

* acouS: snsf,: T^srjssr T a,s may comprise - for « 

successive time interval to price a erie of a 5u2* eTe tZ?™™ ° W ^ ° f 3 Series of 
Means are provided for generating an acoustic ^ mafch scorl ' l9nal \ rep t resenti "9 «"e feature values, 
hypotheses. Each acoustic match score comprises Tn estimate o th ° f * ° f Series 

acoust,c model of the word-series hypothesi S £ 0 f ^T"" ° f 3 matCh between an 

« generate two or more word-series hypothesis signals rLrSntinn T° fe3tUre Si9na,s - Rnal| V- ™ans 
acoustic match scores. 9 re P resen t'ng word-ser.es hypotheses having the best 

The acoustic processor may comprise a microphone 

^='^^2^ one w« ,„ „ teaM „. 
o liquid crystal display. p y ans may be - for example, a cathode ray tube or a 

^fT ^ ca^^ ™ the sound of at least one word in at 

a speech synthesizer and/or a loudspeaker 99nerat,n9 means mav com P ris °. example. 

. mJS.'""" St0n ' n9 WOrd -- ^tneses may comprise an electronic read/write 

The category feature has one of two or more ^ 0^1^. ° f ^ ° r m ° re different va,u es. 

measuring the predictor feature value ^^Z^S^T^ , aPParat " S COmpriSes means for 

Means are provided for selecting one or more difSn, nonfl ^ ^ 6Vent ,n the set of eve "ts. 

and for selecting at least three different noio^ S6tS 0f e "W values, 

category feature values contains .ess than SSSSS^SZ^f featUre V3,ues - Each s « * 

values contains less than a., possible ^Inea^^s ^ ^ m ° f predictor featu ' e 

set of ^rs^j^ss oTcaC^r 9 , a *r> ptMn — - «* 

comprises an estimate of how well the set o f 7re*i^Zli *T T M prediction 
category feature values for all events in th* J* Z ^L I™ ° Va ' U8S pred,cts the occurrence of one set of 




EP 0 555 545 A1 



primary prediction score. 

Means are also provided for generating a secondary prediction score tor each set of predictor feature 
values, other than the set of predictor feature values having the best primary prediction score, and for each 
set of category feature values. Each secondary prediction score comprises an estimate of how well the set 

5 of predictor feature values predicts the occurrence of one set of category feature values for all events in the 
first class of observed events. An event from the first class of events is classified in a first sub-class if the 
predictor feature value of the event is a member of the set of predictor feature values having the bes 
secondary prediction score. 

In the classification apparatus and method according to the invention, each primary prediction score 

10 comprises an estimate of how well the set of predictor feature values predicts the occurrence of a set of 
category feature values in the set of observed events, weighted by a primary weighting factor associated 
with the set of predictor feature values. Each secondary prediction score comprises an estimate of how well 
the set of predictor feature values predicts the occurrence of a set of category feature values in the first 
class of observed events, weighted by a secondary weighting factor associated with the set of predictor 

75 feature values. At least one set of predictor feature values, other than the set of predictor feature values 
having the best primary prediction score, has an associated primary weighting factor which is not equal to 
its associated secondary weighting factor. 

In one aspect of the invention, each event comprises a series of two or more words. 

The apparatus may further comprise means for classifying an event from the first class of events in a 

20 second sub-class if the predictor feature value of the event is not a member of the set of predictor feature 
values having the best secondary prediction score. 

The invention is also a speech recognition apparatus comprising an acoustic processor for outputting a 
sequence of coded representations of an utterance to be recognized. Means are provided for generating an 
acoustic match score for each of a plurality of word-series hypotheses. Each acoustic match score 

25 comprises an estimate of the closeness of a match between an acoustic model of the word-series 
hypothesis and the sequence of coded representations of the utterance. Each word-series hypothesis 
comprises two or more words. Means are also provided for generating two or more candidate word-series 
hypothesis signals representing word-series hypotheses having the best acoustic match scores. A language 
generator according to the present invention receives the candidate word-series hypotheses and outputs at 

30 least one word signal representing at least one word in at least one best candidate word-series hypothesis. 

In the language generator apparatus and method, and the speech recognition apparatus according to 
the present invention, by grouping words in a language model into classes, a highly accurate language 
model can be obtained with a relatively small number of parameters. By combining a number of different 
class-based language models, a highly accurate language model can be obtained from relatively small 

35 amounts of training text. 

In an apparatus and method for classifying a set of observed events according to the present invention, 
by assigning different weighting factors to a set of predictor feature values for different levels of 
classification, the resulting classified word sequences can be used to generate an improved language 
model. 

40 

Brief Description of the Drawing 

Fig. 1 is a block diagram of an example of a language generator according to the present invention. 
Fig. 2 is a block diagram of an example of a word-series hypothesis generator for a language 
45 generator according to the present invention. 

Fig. 3 is a block diagram of an example of a single-unit hypothesis score generator for a language 

generator according to the present invention. 
Fig.4 is a block diagram of an example of an apparatus for classifying a set of observed events 
according to the present invention, 
so Fig. 5 is a block diagram of an example of an acoustic processor for a word-series hypothesis 
generator. 

Figure 1 is a block diagram of an example of a language generator according to the present invention. 
The language generator comprises a word-series hypothesis generator 10 for generating two or more word- 
series hypothesis signals. Each word-series hypothesis signal represents a word-series hypothesis compris- 
55 ing a series of linguistic units. Each linguistic unit comprises one or more words selected from a language. 

One example of a word-series hypothesis generator 10 is shown in the block diagram of Figure 2. In 
this example, the word-series hypothesis generator 10 comprises an acoustic processor 12 for measuring 
the value of at least one feature of an utterance over each of a series of successive time intervals to 



"W^OOCtD- <EP 0555545A1 J_> 



EP 0 555 545 A1 



45 



50 



55 



s acoustic processor 12. sue moaeis store 16) and the ser.es of acoustic feature signals from 

Algorithm. Acoustic match scores may be obS 1 Zl T ?" *° ^rd-Backward 

for example, L.R. Bah., et al, March ! 9* ^ ci^boveT " maXimUm """""^ eSt '' mati ° n - (See ' 

A.t e rn a tive.y, the word-series hypmh^ 
upon automatic language transition scores or oS slZ™" 6 ? WOrd - Series ^P^eses based 
Brown et al. June 1990, and E. Mays et a., 1991 TSUS^ !) ^ ^ eXamp,e ' P R 

" r ™r 9 e^ ^le-unit hypothesis score 

A word-series hypotheses score geirato Soene^ ^ T ^ *" word - series hypothec. 
*> word-series hypothesis. Each word-irE^ a ^d-senes hypothesis score signa. for each 

-it hypothesis scores for the .inguistic «nH.nrS^X2T~ & °' ^ Si " 9le - 

having the best word-series hypotheS Tcores TILTTTS Mnd,d-B hypotheses 

repr ^2^^^ one word si9nai 

-^™gt,™ -erator 20. The Re- 

occurrence of a first class of linguistic uTs C om "La T?TF-« CM ° nal pmbsbm * of 
occurrence of a context comprising the lingu st^ ^'T* linguistic unit " 9 iven »"e 

individual .inguistic unit. Means 30 are pS fo e imat™ 2 ° ,her than ths ,irst 
the first individual linguistic unit given the occurrence S £?, ! C 1 ond,t,onal Probability of occurrence of 
occurrence of the context. occurrence of the first class of linguistic units, and given the 

The conditional probability of occurrence of a first r^e* ■• 
context may be estimated from the traiZ "ex 32 bv couLnn J"* - 9 ' Ven the occ "™<* of the 

text of words in the first class of linguistic units and Lvino I'J, * iT" * occurrences the training 
the number of occurrences in the trainhio text Vn,»l 9 ^selected context, and dividing that count by 
probability of occurrence of £ S * ected ^ext. The conditional 

linguistic units, and given the occurrence of the context mav fZT™* ° f the first class of 

counting the number of occurrences in Xhe^n^eTot7l,^ TT" 0m *" Wn,n8 t6Xt 32 * 
context, and dividing that count by the number nf nl dU3> ' ,n9uistic unit navin 9 the selected 

class having the selected context occurrences in the training text of any word in the first 

hypo^s^^ - T 34 for generating . sing|e . unjt 

a combination of the estimated condition^ comprising 
the occurrence of the context and the estim flto rT^L I . first class of l,n 9uistic units given 

linguistic unit given the occurence JinSSi^^T 1 ^^ ° f ° CCUrr6nCe ° f the first indi ^ a " 

Preferably, me conditions probaS 0, ^TTT - ^ 
occurrence of the first class of linguistic uL an^ * rSt ": d,V,dua, ,in 9 ui *ic unit given the 

the conditional probabHity of 000^^%^,^^^ ° f C ° nteXt may be estimated * 
Cass of linguistic units. In this case, the condtoon^ ^ th8 ° CCUrrence of the ,irst 

unit given the occurrence of the first class of ^ * - ** indiVidUa ' 

counting the number of occurrences in training 3 of ^^the XsT^mh .T^ * 0m ^ trai " in9 text b * 
count by the number of occurrences in the tra* ™ text of TnT r n9U ' StiC Unit> and dividina tha t 
units. n Ine a,nin 9 text of all Imguistoc units in the first class of linguistic 

The conditional probability of occurrence of the first h«« „* i- ' • 
context may be estimated by the conditional «£J5£1?LJ ""guistic un.ts, given the occurrence of the 
given the occurrence of a first context seT Smr^E, n 0CCUrrence of « rat class of linguistic units, 
hypothesis other than the first indi^ualglS unrt 9 °' m ° re UnitS in »• word " s ^ 
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Alternatively, the conditional probability of occurrence of the first class of linguistic units, given the 
occurrence of the context may be estimated as a weighted combination of two or more components. One 
component may be, for example, the conditional probability of occurrence of the first class of linguistic 
units, given the occurrence of a first context set comprising one or more linguistic units in the word-series 

5 hypothesis other than the first individual linguistic unit. A second component may be the conditional 
probability of occurrence of the first class of linguistic units, given the occurrence of a second context set, 
different from the first context set, comprising one or more linguistic units in the word-series hypothesis 
other than the first individual linguistic unit. The weighting factors may be obtained, for example, by the 
method of deleted interpolation. (See, for example, L.R. Bahl, et at, March 1983, cited above.) 

10 In another alternative single-unit hypothesis generator 20 shown in Figures 1 and 3, the means 28 may 
also estimate the conditional probability of occurrence of a second class of linguistic units comprising the 
first individual linguistic unit, given the occurrence of the context comprising the linguistic units in the word- 
series hypothesis other than the first individual linguistic unit. The means 30 may also estimate the 
conditional probability of occurrence of a first individual linguistic unit given the occurrence of the second 

15 class of linguistic units, and given the occurrence of the context. Both conditional probabilities may be 
estimated from the training text 32, as described above. 

Means 34 then generate a second component score comprising a combination of the estimated 
conditional probability of occurrence of the second class of linguistic units given the occurrence of the 
context, and the estimated conditional probability of occurrence of the first individual linguistic given the 

20 occurrence of the second class of linguistic units and given the occurrence of the context. A single-unit 
hypothesis score for the first individual linguistic unit comprises a weighted combination of the first 
component score and the second component score. The weighting factors can be obtained, for example, by 
the method of deleted interpolation. 

The conditional probability of occurrence of the first individual linguistic unit given the occurrence of the 

25 second class of linguistic units, and given the occurrence of the context may be estimated by the 
conditional probability of occurrence of the first individual linguistic unit given the occurrence of the second 
class of linguistic units. The conditional probability of occurrence of the first individual linguistic unit given 
the occurrence of the second class of linguistic units may be estimated from the training text 32 by 
counting the number of occurrences in the training text of the first individual linguistic unit, and by dividing 

30 that count by the number of occurrences in the training text of linguistic units in the second class of 
linguistic units. 

The conditional probability of occurrence of the second class of linguistic units, given the occurrence of 
the context may be estimated as the conditional probability of occurrence of the second class of linguistic 
units, given the occurrence of a first context set comprising one or more linguistic units in the word-series 

35 hypothesis other than the first individual linguistic unit. Alternatively, the conditional probability of the 
second class of linguistic units, given the occurrence of the context may be estimated as a weighted 
combination of two or more terms such as (a) the conditional probability of occurrence of the second class 
of linguistic units given the occurrence of a first context set comprising one or more linguistic units in the 
word-series hypothesis other than the first individual linguistic unit, and (b) the conditional probability of 

40 occurrence of the second class of linguistic units given the occurrence of a second context set, different 
from the first context set, comprising one or more linguistic units in the word-series hypothesis other than a 
first individual linguistic unit. The weighted coefficients may be obtained, for example, by the method of 
deleted interpolation. 

The operation of the language generator apparatus and method according to the present invention may 
45 be illustrated by the following hypothetical example. 

Table 1 is a hypothetical example of seven classes A through F of linguistic units, where each linguistic 
unit is a single word. 

50 
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TABLE 1 



w 



CLASS 


WORDS 


A 
B 
C 
D 
E 
F 


fast, speedy, hungry, hunting, angry, deadly 

train, house, car, book, computer, ski 

on, of, in, as, near, for 

my, his, a, the, what, some 

lion, hawk, shark, wolf, tiger, bear 

air, forest, sky, mountain, park, tree 



15 



t° h rr nc ^ th ^ 01 the conditionai « 

b. oht °1 1" 6 Pri ° r W ° rd W(M > in th * series * dScusTed a ^ ^i"™ 06 ° f * COm ? risin 9 
be obtained by counting occurrences of words in a WrtnJS ' Conditional Probabilities may 
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Class 

of 
W(i-l) 

A 
B 
C 
D 
E 
F 



W(i) 



TABLE 2 

Prob [Class of W(i) | class of W(i-l)i 
C D E 



0.047 
0.013 
0.0588 
0.0224 
0.0093 
0.0084 



0.0208 
0.0005 
0.0291 
0.0208 
0.0356 
0.0448 



0.025 
0.026 
0.0043 
0.0499 
0.024 
0.0448 



0.0236 
0.0466 
0.0291 
0.0006 
0.0651 
0.0474 



0.0869 
0.0147 
0.0269 
0.0159 
0.0004 
0.0397 



0.0028 
0.0262 
0.0673 
0.0022 
0.034 
0.0003 



35 



W< ^rse^ of occurrence of the Cass of the word 

W(i-2) in the word-series hypothesis ^Sd^'^T^ ^ ^ * ^ ^ ^ 
rences of words in a training text. «***onal probab.ht.es may be obtained by counting occur- 
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Class 




of 


W(i) A 


W(i-2) 


A 


0.0016 


B 


0.0518 


C 


0.034 


D 


0.021 


E 


0.0086 


F 


0.0194 



TABLE 3 

Proyciass of W(i, | C las S of W (i- 2 )] 
c D e 



0.0049 
0.0432 
0.041 
0.0324 
0.0232 
0.0157 



0.047 
0.0011 
0.0318 
0.0443 
0.0405 
0.0318 



0.0038 
0.0313 
0.0291 
0.0076 
0.0383 
0.0281 



0.0113 
0.0135 
0.0405 
0.0421 
0.0005 
0.0291 



0.0513 
0.007 

0.0329 

0.0518 
0.041 

0.0475 



55 



occu^encT^^ va.ues for the conditional probability of 

containing the word W(i). S6neS h ™ oih ™s g.ven the occurrence of the class C[W(i)] 
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TABLE 4 





Prob [W(i)l Class of W(i)] 


5 


W(i) 


fast 


speedy 


hungry 


hunting 


angry 


deadly 




P{W(i)IC[W(i)]} 


0.224 


0.198 


0.407 


0.041 


0.033 


0.097 




W(i) 


train 


house 


car 


book 


computer 


ski 




P{W(i)IC[W(i)]} 


0.336 


0.179 


0.294 


0.01 


0.006 


0.175 




W(i) 


on 


of 


in 


as 


near 


for 


10 


P{W(i)IC[W(i)]} 


0.134 


0.142 


0.167 


0.064 


0.297 


0.196 




W(i) 


my 


his 


a 


the 


what 


some 




P{W(i)IC[W(i)]} 


0.177 


0.159 


0.083 


0.198 


0.181 


0.201 




W(i) 


lion 


hawk 


shark 


wolf 


tiger 


bear 




P{W(i)1C[W(i)]} 


0.184 


0.212 


0.241 


0.071 


0.027 


0.265 


15 


W(i) 


air 


forest 


sky 


mountain 


park 


tree 




P{W(i)IC[W(i)]} 


0.249 


0.123 


0.145 


0.055 


0.257 


0.17 



Table 5 shows the single-unit hypothesis scores for the words Wj in four different word-series 
hypotheses. The single-unit hypothesis scores were obtained using the conditional probability values of 
Tables 2, 3, and 4. In the hypothetical example, the conditional probabilities of the class of W(i) given the 
class of the prior word and given the class of the second prior word are weighted by 0.75 and 0.25, 
respectively. 
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Com- 


Com- 


Combin- 


30 


H i-2 


w i- i 
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PfC^MCtWj _ ,)) 






ponent 


ponent 


ation 




the 


angry 


shark 


0.08G9 


0.0421 


0.241 


0.0209 


0.0101 


0.0162 




the 


angry 


park 


0.0028 


0.0516 


0.257 


0.0007 


0.0133 


0.0039 




near 


the 


ski 


0.02O0 


0.041 


0. 175 


0.0036 


0.0072 


0.0045 




near 


the 


sky 


0.0022 


0.0329 


0. 145 


0.0003 


0.0048 


0.0014 


35 












Weight « 


0.75 


0.25 





In a similar manner, the single-unit hypothesis scores may be obtained for each other word in each 
hypothesis. Thereafter, the word-series hypothesis score for each hypothesis is obtained by combining the 
single-unit hypothesis scores for all words in the hypothesis. When the single-unit scores represent 
estimated probabilities, the scores may be combined by multiplication. When the single-unit scores 
represent estimated logarithms of probabilities, the scores may be combined by addition. 

Returning to the hypothetical examples of Table 5. as between the word series, "the angry shark" and 
"the angry park", the single-unit hypothesis scores of the first two words will be the same for both 
hypotheses. Therefore, the word-series hypothesis score for "the angry shark" will be better (in this case 
higher is better) than the word-series hypothesis score for "the angry park". Consequently, as between the 
word series, "the angry shark" and "the angry park", "the angry shark" will be displayed. 

Similarly, as between the word series, "near the ski" and "near the sky," the single-unit hypothesis 
scores of the first two words will be the same for both hypotheses. Therefore, the word-series hypothesis 
score for "near the ski" . will be better (in this case higher is better) than the word-series hypothesis score 
for "near the sky". Consequently, as between the word series, "near the ski" and "near the sky", "near the 
ski" will be displayed. 

The classes of linguistic units, and the context sets comprising one or more linguistic units for the 
language generator method and apparatus according to the present invention can be obtained by an 
apparatus and method for classifying a set of observed events. In the set of observed events, each event 
has a predictor feature and a category feature. The predictor feature has one of two or more different 
possible values. The category feature has one of two or more different possible values. 
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n n 



P(d) P(Cn) 
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where P(ci , o% ) is the probability that in a series of two adjacent words {w 1t w 2 }, the first word is in class 
c m and the second word is in class c n . 

The following algorithm may be used to partition a vocabulary of V words into two subsets such that the 

10 average mutual information of subsets of adjacent words is high. Initially, each word of the vocabulary V is 
assigned to a distinct subset, and the average mutual information between subsets of adjacent words is 
obtained from Equation 3. For every possible pair of subsets, that one pair of subsets is merged, and the 
average mutual information between subsets of adjacent words is calculated. The pair of subsets for which 
the loss in average mutual information is least is then merged, so only (V-1) subsets remain. 

75 From the (V-1) subsets, each word of the vocabulary is moved to the subset which results in a partition 
having the greatest mutual information. When there is no reassignment of a word which leads to a partition 
with greater average mutual information, the (V-1) subsets are complete. While it may be possible to find a 
partition with higher average mutual information by simultaneously reassigning two or more words, such a 
search is regarded as too costly to be feasible. 

20 Thereafter, the process is repeated to merge the (V-1) subsets into (V-2) subsets, and so on, until only 
two subsets remain. 

In the merging algorithm described above, the order in which the clusters are merged defines a binary 
tree. The root of the binary tree is the entire vocabulary of words. The leaves of the binary tree are the 
individual words in the vocabulary. Individual nodes of the binary tree define additional vocabulary subsets 

25 which may be used to determine sets of category feature values and sets of predictor feature values. 

Returning to Figure 4, a primary prediction score generator 44 generates a primary prediction score for 
each set of predictor feature values from set selector 42 and for each set of category feature values from 
set selector 40. Each primary prediction score comprises an estimate of how well the set of predictor values 
predicts the occurrence of one set of category feature values for all events in the set of observed events 38. 

30 Means 46 classify an event in a first class if the predictor feature value of the event is a member of the set 
of predictor feature values having the best primary prediction score. 

A secondary prediction score generator 48 generates a secondary prediction score for each set of 
predictor feature values other than the set of predictor feature values having the best primary prediction 
score, and for each set of category feature values. Each secondary prediction score comprises an estimate 

35 of how well the set of predictor feature values predicts the occurrence of one set of category feature values 
for all events in the first class of observed events 46. An event from a first class of events is classified in a 
first subclass 50 if the predictor feature value is a member of the set of predictor feature values having the 
best secondary prediction score. 

Each primary prediction score comprises an estimate of how well the set of predictor feature values 

40 predicts the occurrence of a set of category feature values in the set of observed events, weighted by a 
primary weighting factor associated with the set of predictor feature values. Each secondary prediction 
score comprises an estimate of how well the set of predictor feature values predicts the occurrence of a set 
of category feature values in the first class of observed events, weighted by a secondary weighting factor 
associated with the set of predicted feature values. 

45 At least one set of predictor feature values, other than the set of predictor feature values having the best 
primary prediction score, has an associated primary weighting factor which is not equal to its associated 
secondary weighting factor. 

The apparatus and method for classifying a set of observed events may further comprise means for 
classifying an event from the first class of events in a second subclass if the predictor feature value of the 

so event is not a member of the set of predicted feature values having the best secondary prediction score. 

The prediction score for a set B of predictor feature values and for a set Y of category feature values 
may be, for example, the conditional entropy H(Y|B) of a binary split of the events according to the set of 
category feature values given a binary split of events according to the set of predictor feature values. 
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H(Y I B) = - PfY . B) I nn PfY 1 n\ 

- P(Y, B) log P(Y|B) 

-P(Y, B)logP(Y|B) [4] 

- P(Y, B) log P(Y |B) 

io In Equation 7 is the complement of the set YandBi, th« ™ . 
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An alternative prediction score may be the cross-entropy 

H,(Y,B) = - P( y /B)logM(Y(B) 
~ P(?, B) log M(Y|B) 
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- P(Y, B) logM(Y|B) 
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- P(Y, B, B'J log P(Y | B, B'J 

- P(Y , B, B'J log P(Y | B, B'J [6] 

- P(Y , B , B'J log P(Y | B , B'J 

- P(*~, B, B'J log P(Y|B, B'J 
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series hypothesis and other sLge ~ — 
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One example of an acoustic processor 12 (Figure 2) is shown in Figure 5. The measuring means 
includes a microphone 52 for generating an analog electrical signal corresponding to the utterance. The 
analog electrical signal from microphone 52 is converted to a digital electrical signal by analog to digital 
converter 54. For this purpose, the analog signal may be sampled, for example, at a rate of twenty kilohertz 
5 by the analog to digital converter 54. 

A window generator 56 obtains, for example, a twenty millisecond duration sample of the digital signal 
from analog to digital converter 54 every ten milliseconds (one centisecond). Each twenty millisecond 
sample of the digital signal is analyzed by spectrum analyzer 58 in order to obtain the amplitude of the 
digital signal sample in each of, for example, twenty frequency bands. Preferably, spectrum analyzer 58 
io also generates a twenty-first dimension signal representing the total amplitude or total power of the twenty 
millisecond digital signal sample. The spectrum analyzer 58 may be, for example, a fast Fourier transform 
processor. Alternatively, it may be a bank of twenty band pass filters. 

The twenty-one dimension vector signals produced by spectrum analyzer 58 may be adapted to 
remove background noise by an adaptive noise cancellation processor 60. Noise cancellation processor 60 
15 subtracts a noise vector N(t) from the feature vector F(t) input into the noise cancellation processor to 
produce an output feature vector F(t). The noise cancellation processor 60 adapts to changing noise levels 
by periodically updating the noise vector N(t) whenever the prior feature vector F(t-1) is identified as noise 
or silence. The noise vector N(t) is updated according to the formula 

20 N(t) = N(t-1) + k[F(t-1) - Fp(t-1)]. [7] 

where N(t) is the noise vector at time t, N(t-1) is the noise vector at time (t-1). k is a fixed parameter of the 
adaptive noise cancellation model. F(t-1) is the feature vector input into the noise cancellation processor 60 
at time (t-1) and which represents noise or silence, and Fp(t-1) is one silence or noise prototype vector. 

25 from store 62, closest to feature vector F(t-1). 

The prior feature vector F(t-1) is recognized as noise or silence if either (a) the total energy of the 
vector is below a threshold, or (b) the closest prototype vector in adaptation prototype vector store 64 to the 
feature vector is a prototype representing noise or silence. For the purpose of the analysis of the total 
energy of the feature vector, the threshold may be, for example, the fifth percentile of all feature vectors 

30 (corresponding to both speech and silence) produced in the two seconds prior to the feature vector being 
evaluated. 

After noise cancellation, the feature vector F(t) is normalized to adjust for variations in the loudness of 
the input speech by short term mean normalization processor 66. Normalization processor 66 normalizes 
the twenty-one dimension feature vector P(t) to produce a twenty dimension normalized feature vector X(t). 
35 The twenty-first dimension of the feature vector F(t), representing the total amplitude or total power, is 
discarded. Each component i of the normalized feature vector X(t) at time t may, for example, be given by 
the equation 

Xi(t) = P,(t) - Z(t) [8] 

40 

in the logarithmic domain, where F\(t) is the i-th component of the nonnormalized vector at time t, and 
where Z(t) is a weighted mean of the components of P(t) and Z(t - 1) according to Equations 9 and 10: 

Z(t) = 0.9 Z(t-1) + 0.1 M(t) [9] 

45 

and where 



50 M(t) = ^F\(t) [10] 

i 

The normalized twenty dimension feature vector X(t) may be further processed by an adaptive labeler 68 to 
55 adapt to variations in pronunciation of speech sounds. An adapted twenty dimension feature vector X*(t) is 
generated by subtracting a twenty dimension adaptation vector A(t) from the twenty dimension feature 
vector X(t) provided to the input of the adaptive labeler 68. The adaptation vector A(t) at time t may, for 
example, be given by the formula 
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A(t) = A(t-1) + k[X(t-1) - Xp(t-1)l. fill 
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means for outputting at least one word signal representing at least one word in at least one best 
candidate word-series hypothesis; 

characterized in that the means for generating a single-unit hypothesis score signal for a first individual 
linguistic unit comprises: 

5 means for estimating the conditional probability of occurrence of a first class of linguistic units 

comprising the first individual linguistic unit, given the occurrence of a context comprising the linguistic 
units in the word-series hypothesis other than the first individual linguistic unit; 

means for estimating the conditional probability of occurrence of the first individual linguistic unit given 
the occurrence of the first class of linguistic units, and given the occurrence of the context; and 
10 means for generating a single-unit hypothesis score signal for the first individual linguistic unit 
comprising a first component score comprising a combination of the estimated conditional probability of 
occurrence of the first class of linguistic units given the occurrence of the context, and the estimated 
conditional probability of occurrence of the first individual linguistic unit given the occurrence of the first 
class and given the occurrence of the context. 

15 

2. A language generator as claimed in Claim 1, characterized in that the conditional probability of 
occurrence of the first individual linguistic unit given the occurrence of the first class of linguistic units, 
and given the occurrence of the context is estimated by the conditional probability ofoccurrence of the 
first individual linguistic unit given the occurrence of the first class of linguistic units. 

20 

3. A language generator as claimed in Claim 1, characterized in that the conditional probability of 
occurrence of the first class of linguistic units, given the occurrence of the context is estimated by the 
conditional probability of occurrence of the first class of linguistic units, given the occurrence of a first 
context set comprising one or more linguistic units in the word-series hypothesis other than the first 

25 individual linguistic unit 

4. A language generator as claimed in Claim 1. characterized in that the conditional probability of 
occurrence of the first class of linguistic units, given the occurrence of the context is estimated as a 
weighted combination of: 

30 the conditional probability of occurrence of the first class of linguistic units, given the occurrence of a 
first context set comprising one or more linguistic units in the word-series hypothesis other than the 
first individual linguistic unit; and 

the conditional probability of occurrence of the first class of linguistic units, given the occurrence of a 
second context set, different from the first context set, comprising one or more linguistic units in the 
35 word-series hypothesis other than the first individual linguistic unit. 

5. A language generator as claimed in Claim 1, characterized in that the means for generating a single- 
unit hypothesis score signal for the first individual linguistic unit further comprises: 

means for estimating the conditional probability of occurrence of a second class of linguistic units 
40 comprising the first individual linguistic unit, given the occurrence of the context comprising the 
linguistic units in the word-series hypothesis other than the first individual linguistic unit; 
means for estimating the conditional probability of occurrence of the first individual linguistic unit given 
the occurrence of the second class of linguistic units, and given the occurrence of the context; 
means for generating a second component score comprising a combination of the estimated conditional 
45 probability of occurrence of the second class of linguistic units given the occurrence of the context, and 
the estimated conditional probability of occurrence of the first individual linguistic unit given the 
occurrence of the second class of linguistic units and given the occurrence of the context; and 
means for generating a single-unit hypothesis score for the first individual linguistic unit comprising a 
weighted combination of the first component score and the second component score. 

50 

6- A language generator as claimed in Claim 5, characterized in that the conditional probability of 
occurrence of the first individual linguistic unit given the occurrence of the second class of linguistic 
units, and given the occurrence of the context is estimated by the conditional probability of occurrence 
of the first individual linguistic unit given the occurrence of the second class of linguistic units. 

55. 

7. A language generator as claimed in Claim 5, characterized in that the conditional probability of 
occurrence of the second class of linguistic units, given the occurrence of the context is estimated by 
the conditional probability of occurrence of the second class of linguistic units, given the occurrence of 

15 
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storing one or more candidate word-series hypothesis signals representing candidate word-series 
hypotheses having the best word-series hypothesis scores; and 

outputting at least one word signal representing at least one word in at least one best candidate word- 
series hypothesis; 

5 characterized in that the step of generating a single-unit hypothesis score signal for a first individual 

linguistic unit comprises: 

estimating the conditional probability of occurrence of a first class of linguistic units comprising the first 
individual linguistic unit, given the occurrence of a context comprising the linguistic units in the word- 
series hypothesis other than the first individual linguistic unit; 

10 estimating the conditional probability of occurrence of the first individual linguistic unit given the 
occurrence of the first class of linguistic units, and given the occurrence of the context; and 
generating a single-unit hypothesis score signal for the first individual linguistic unit comprising a first 
component score comprising a combination of the estimated conditional probability of occurrence of 
the first class of linguistic units given the occurrence of the context, and the estimated conditional 

75 probability of occurrence of the first individual linguistic unit given the occurrence of the first class and 
given the occurrence of the context. 

19. A method as claimed in Claim 18, characterized in that the conditional probability of occurrence of the 
first individual linguistic unit given the occurrence of the first class of linguistic units, and given the 

20 occurrence of the context is estimated by the conditional probability of occurrence of the first individual 

linguistic unit given the occurrence of the first class of linguistic units. 

20. A method as claimed in Claim 18, characterized in that the conditional probability of occurrence of the 
first class of linguistic units, given the occurrence of the context is estimated by the conditional 

25 probability of occurrence of the first class of linguistic units, given the occurrence of a first context set 
comprising one or more linguistic units in the word-series hypothesis other than the first individual 
linguistic unit. 

21. A method as claimed in Claim 18, characterized in that the conditional probability of occurrence of the 
30 first class of linguistic units, given the occurrence of the context is estimated as a weighted combination 

of: 

the conditional probability of occurrence of the first class of linguistic units, given the occurrence of a 
first context set comprising one or more linguistic units in the word-series hypothesis other than the 
first individual linguistic unit; and 
35 the conditional probability of occurrence of the first class of linguistic units, given the occurrence of a 
second context set, different from the first context set, comprising one or more linguistic units in the 
word-series hypothesis other than the first individual linguistic unit. 

22. A method as claimed in Claim 18, characterized in that the step of generating a single-unit hypothesis 
40 score signal for the first individual linguistic unit further comprises: 

estimating the conditional probability of occurrence of a second class of linguistic units comprising the 
first individual linguistic unit, given the occurrence of the context comprising the linguistic units in the 
word-series hypothesis other than the first individual linguistic unit; 

estimating the conditional probability of occurrence of the first individual linguistic unit given the 
45 occurrence of the second class of linguistic units, and given the occurrence of the context; 

generating a second component score comprising a combination of the estimated conditional probabil- 
ity of occurrence of the second class of linguistic units given the occurrence of the context, and the 
estimated conditional probability of occurrence of the first individual linguistic unit given the occurrence 
of the second class of linguistic units and given the occurrence of the context; and 
so generating a single-unit hypothesis score for the first individual linguistic unit comprising a weighted 
combination of the first component score and the second component score. 

23. A method as claimed in Claim 22, characterized in that the conditional probability of occurrence of the 
first individual linguistic unit given the occurrence of the second class of linguistic units, and given the 

55 occurrence of the context is estimated by the conditional probability of occurrence of the first individual 

linguistic unit given the occurrence of the second class of linguistic units. 
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primary weighting factor associated with the set of predictor feature values; 

each secondary prediction score comprises an estimate of how well the set of predictor feature values 
predicts the occurrence of a set of category feature values in the first class of observed events, 
weighted by a secondary weighting factor associated with the set of predictor feature values; and 
5 at least one set of predictor feature values, other than the set of predictor feature values having the 

best primary prediction score, has an associated primary weighting factor which is not equal to its 
associated secondary weighting factor. 

30. An apparatus as claimed in Claim 29, characterized in that each event comprises a series of two or 
10 more words. 

31- An apparatus as claimed in Claim 29, further comprising means for classifying an event from the first 
class of events in a second subclass if the predictor feature value of the event is not a member of the 
set of predictor feature values having the best secondary prediction score. 

75 

32. A method of classifying a set of observed events, each event having a predictor feature and a category 
feature, said predictor feature having one of two or more different possible values, said category feature 
having one of two or more different possible values, said method comprising the steps of: 
measuring the predictor feature value and the category feature value of each event in the set of events; 

20 selecting one or more different, noncomplementary sets of category feature values, each set containing 

less than all possible category feature values; 

selecting at least three different, noncomplementary sets of predictor feature values, each set of 
predictor feature values containing less than all possible predictor feature values; 

generating a primary prediction score for each set of predictor feature values and for each set of 
25 category feature values, each primary prediction score comprising an estimate of how well the set of 
predictor feature values predicts the occurrence of one set of category feature values for all events in 
the set of observed events; 

classifying an event in a first class if the predictor feature value of the event is a member of the set of 
predictor feature values having the best primary prediction score; 
30 generating a secondary prediction score for each set of predictor feature values, other than the set of 
predictor feature values having the best primary prediction score, and for each set of category feature 
values, each secondary prediction score comprising an estimate of how well the set of predictor feature 
values predicts the occurrence of one set of category feature values for all events in the first class of 
observed events; and 

35 classifying an event from the first class of events in a first subclass if the predictor feature value of the 
event is a member of the set of predictor feature values having the best secondary prediction score; 
characterized in that: 

each primary prediction score comprises an estimate of how well the set of predictor feature values 
predicts the occurrence of a set of category feature values in the set of observed events, weighted by a 

40 primary weighting factor associated with the set of predictor feature values; 

each secondary prediction score comprises an estimate of how well the set of predictor feature values 
predicts the occurrence of a set of category feature values in the first class of observed events, 
weighted by a secondary weighting factor associated with the set of predictor feature values; and 
at least one set of predictor feature values, other than the set of predictor feature values having the 

45 best primary prediction score, has an associated primary weighting factor which is not equal to its 
associated secondary weighting factor. 

33. A method as claimed in Claim 32, characterized in that each event comprises a series of two or more 
words. 

50 

34. A method as claimed In Claim 32, further comprising the step of classifying an event from the first 
class of events in a second subclass if the predictor feature value of the event is not a member of the 
set of predictor feature values having the best secondary prediction score. 

i 

55 35. A speech recognition apparatus comprising: 

an acoustic processor for outputting a sequence of coded representations of an utterance to be 
recognized; 

means for generating an acoustic match score for each of a plurality of word-series hypotheses, each 
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